Eric has a plan to add all 8 arrow directions instead of just the "keep left" and "keep right" directions on the metal standing board. I'm not sure if that does all you want or have a better idea. Or if it's important to have arrows on the alphanumeric boards.
We are actually right on the question of alphanumeric boards now. With 36 slots used for A-Z and 0-9 there are 12 spare slots.
We have proposed use for these 12 "spare" slots but I want to ask you if some swaps would be better.
BOARD1: A B C D E F G H I J K L M _ < >
BOARD2: N O P Q R S T U V W X Y Z ! & @
BOARD3: 0 1 2 3 4 5 6 7 8 9 # * % + - _
That's what Eric suggested. I guess one of the _ is a actually a SPACE. I don't know if < and > should actually be a triangle or arrow instead? And are there any better choices than & ! + - % * _ ?
I suppose to my mind the most useful extras are: SPACE # < > @ (not sure about the others)
Thanks for the feedback.
Let me try to make sure we are clear by asking some questions:
I think you are saying: Polystyrene, movable equivalent of Banner1 and Banner2? And when you say the pics file, you mean like AX_ADS1.jpg or equivalent?
The countdown boards, do you mean the distance to corner markers? And so what you are suggesting is on the back of them could be mapped another slot from an AX_ADSx file?
Good suggestion but I'll say not for this update. To get into InSim coding now is way too complicated for me as I've got a few things on the go. Just adding a few objects is about all we can do for now.
But either during public beta testing or maybe for a version after the 'big release' would be a reasonable time.
Interesting idea. Let's see if Eric like the idea.
I think you mean, massive flat chalk letters? I supposed they should be extended "vertically" so they are easy to read from a car like painted SLOW sign on a road?
What size would you suggest? Something like a SLOW sign?
https://s0.geograph.org.uk/geophotos/02/52/55/2525585_aa2c8f9d.jpg
Let me try to make sure we are clear by asking some questions:
I think you are saying: Polystyrene, movable equivalent of Banner1 and Banner2? And when you say the pics file, you mean like AX_ADS1.jpg or equivalent?
The countdown boards, do you mean the distance to corner markers? And so what you are suggesting is on the back of them could be mapped another slot from an AX_ADSx file?
Good suggestion but I'll say not for this update. To get into InSim coding now is way too complicated for me as I've got a few things on the go. Just adding a few objects is about all we can do for now.
But either during public beta testing or maybe for a version after the 'big release' would be a reasonable time.
Interesting idea. Let's see if Eric like the idea.
I think you mean, massive flat chalk letters? I supposed they should be extended "vertically" so they are easy to read from a car like painted SLOW sign on a road?What size would you suggest? Something like a SLOW sign?
https://s0.geograph.org.uk/geophotos/02/52/55/2525585_aa2c8f9d.jpg
We are currently spending a few days improving layout objects as the old ones are just too ugly. It might not be absolutely necessary before the release but we sort of fell into it, anyway it'll be done soon and had to be done at some point!
We've now got the ability to select up to 16 mappings per object, so for example we could have alphabet boards like this. Maybe a rectangle, with one letter per board.
- 1 board with letters A to M selectable
- 1 board with letters N to Z selectable
- 1 board with numbers 0 to 9 selectable
That only takes 3 object slots - not 36!
I suggest a plain (white by default) polystyrene rectangle that could stand on the ground or be floating. It could also have colours selectable.
Eric asked what size the board should be. Any ideas? I suppose not too big as you want to make words with them. But big enough to be seen from a reasonable distance.
EDIT: How about, similar size to the corner markers, but 1m high and 80cm wide, reasonable proportions for letters?
Last edited by Scawen, .
Another 12 days have passed and some of you may be interested to read the log. The super short version is I've been fixing things, doing things that needed to be done, getting rid of lists of things to do. I'm trying to get all the important things out of the way ready to focus on tyre physics. Things that are less important, I plan to do after the release (apart from the odd thing here or there that I simply feel like doing).
The attached images show what tyre smoke looks like in a tunnel with and without a lightmap!
Log of changes:
Some small tasks to finish the delayed texture loading system
- updated per-frame task sequencer so if a car model was built a texture is not loaded
- renamed / reorganised some of the new functions to make more sense when reading code
- prevented delayed texture loading in modeller and vehicle editor (prefer instant)
- fixed single frame with headless driver standing up when exiting animation editor
- the option to switch between compressed and full skins now uses new reload system
- used MP replay to test switching between high and low resolution auto-download skins
- added critical section to make sure DDS saving cannot coincide with loading DDS skin
Spent a day on Fern Bay in the editor - various fixes (not creative work)
- unlike other tracks, Eric does not plan major Fern Bay updates in the near future
- there were shadow holes due to buildings and footbridges without top surfaces
- as an old favourite I wanted to see it in good condition suitable for release
- it's good for me to spend a little time in the track editor occasionally
Fixed minor issue: players joining in entry screen could trigger texture purge events
- host and guests did a quick load and delete of the new player's car to check setup
- as this car was loaded in "in-game" mode it would trigger a check for unused textures
- but cars no longer point to a setup in the player so this seemed to have no purpose
- so instead of preventing the cars triggering a texture purge, simply avoid this code
Off-topic interface update: Provide a left and right arrow for multi-state options
- it bugged me that to switch off replay saving I had to pass through an extra state
- now I can simply click left arrow to switch off saving / right arrow to enable
- the arrow button is greyed out if you reach the last option in that direction
- the 'greyed out' logic was easily applied to option slider bars so did that too
- tempting to do it for the integer nudge buttons but would have to change all calls
- too much to do so will forget that until the inconsistency bugs me in the future
Some development version updates
- new feature to help search for textures in the editor (in objects and world mappings)
- fixed system to load most recent version of a track for a lesson (not exact match)
- update to allow replays to load most recent version of a track (or currently loaded)
- fixed a thread protection notification exiting from editor using the window X button
Training lesson fixes
- fixed start time of training lessons which were always at midnight on 1 January 1601
- fixed some small issues around loading tracks that triggered thread warning messages
- fixed a "Game call from main thread" thread warning message during training lessons
General fixes
- fix for a thread protection notification when LFS does a full D3D recovery
(I have not seen a D3D recovery other than by deliberately forcing it to happen)
Fix for a problem noticed in MPR but would affect multiplayer generally
- symptom was a puff of smoke when clicking timeline to a certain point in the replay
- took a while to track down the cause (depended on where car was before fast forward)
- related to tyre's 'previous ground position' which is compared with current position
- this previous position was wrong if car was at a different angle before pos packet
- this has very minimal difference if pos packet is for a car that is currently viewed
- bigger difference if moving to a car that has not been viewed (physics not updated)
- solution is to get a good estimate for previous position using velocity and rotation
- testing shows that car velocity gives a good estimate so will leave out rotation now
The problem described above took me into angular momentum, rotation, position updates
- some things were still waiting since the 1000Hz updates (on my notes for months)
- now there is no sub-update, some things can be rearranged a little (for accuracy)
- a position accumulator is needed for low speeds (to avoid a 'minimum speed' issue)
- seems good to be pulling me back into physics as the tyre updates are waiting
- converted old variable FloatPos (was related to sub-updates) to position accumulator
- now super-slow speeds are possible and car cannot be stuck on 'digital rails' (xyz)
Have noticed an issue for some time - physics objects resisted starting to move
- seems to be related to 1000Hz update: object velocity not fast enough after 1 frame
- fixed by reducing the speed at which an object is considered to have started moving
Cleaned up code from recent changes / discarded finished notes / browsed old notes
- copied a few things from the old notes onto a short list of tasks to do for release
- although we are still a way from there I want to know what remains other than tyres
- random OT: made arrow keys work to move between options screens (seems like should)
- old notes reminded me that smoke particles do not pick up lighting from lightmap
- the lightmap stores info about artificial light and sky visibility, in an octree
- that means smoke looks black in the night and also too bright in tunnel during day
- made smoke particles read lightmap - see the attached images for before and after
The attached images show what tyre smoke looks like in a tunnel with and without a lightmap!
Log of changes:
Some small tasks to finish the delayed texture loading system
- updated per-frame task sequencer so if a car model was built a texture is not loaded
- renamed / reorganised some of the new functions to make more sense when reading code
- prevented delayed texture loading in modeller and vehicle editor (prefer instant)
- fixed single frame with headless driver standing up when exiting animation editor
- the option to switch between compressed and full skins now uses new reload system
- used MP replay to test switching between high and low resolution auto-download skins
- added critical section to make sure DDS saving cannot coincide with loading DDS skin
Spent a day on Fern Bay in the editor - various fixes (not creative work)
- unlike other tracks, Eric does not plan major Fern Bay updates in the near future
- there were shadow holes due to buildings and footbridges without top surfaces
- as an old favourite I wanted to see it in good condition suitable for release
- it's good for me to spend a little time in the track editor occasionally
Fixed minor issue: players joining in entry screen could trigger texture purge events
- host and guests did a quick load and delete of the new player's car to check setup
- as this car was loaded in "in-game" mode it would trigger a check for unused textures
- but cars no longer point to a setup in the player so this seemed to have no purpose
- so instead of preventing the cars triggering a texture purge, simply avoid this code
Off-topic interface update: Provide a left and right arrow for multi-state options
- it bugged me that to switch off replay saving I had to pass through an extra state
- now I can simply click left arrow to switch off saving / right arrow to enable
- the arrow button is greyed out if you reach the last option in that direction
- the 'greyed out' logic was easily applied to option slider bars so did that too
- tempting to do it for the integer nudge buttons but would have to change all calls
- too much to do so will forget that until the inconsistency bugs me in the future
Some development version updates
- new feature to help search for textures in the editor (in objects and world mappings)
- fixed system to load most recent version of a track for a lesson (not exact match)
- update to allow replays to load most recent version of a track (or currently loaded)
- fixed a thread protection notification exiting from editor using the window X button
Training lesson fixes
- fixed start time of training lessons which were always at midnight on 1 January 1601
- fixed some small issues around loading tracks that triggered thread warning messages
- fixed a "Game call from main thread" thread warning message during training lessons
General fixes
- fix for a thread protection notification when LFS does a full D3D recovery
(I have not seen a D3D recovery other than by deliberately forcing it to happen)
Fix for a problem noticed in MPR but would affect multiplayer generally
- symptom was a puff of smoke when clicking timeline to a certain point in the replay
- took a while to track down the cause (depended on where car was before fast forward)
- related to tyre's 'previous ground position' which is compared with current position
- this previous position was wrong if car was at a different angle before pos packet
- this has very minimal difference if pos packet is for a car that is currently viewed
- bigger difference if moving to a car that has not been viewed (physics not updated)
- solution is to get a good estimate for previous position using velocity and rotation
- testing shows that car velocity gives a good estimate so will leave out rotation now
The problem described above took me into angular momentum, rotation, position updates
- some things were still waiting since the 1000Hz updates (on my notes for months)
- now there is no sub-update, some things can be rearranged a little (for accuracy)
- a position accumulator is needed for low speeds (to avoid a 'minimum speed' issue)
- seems good to be pulling me back into physics as the tyre updates are waiting
- converted old variable FloatPos (was related to sub-updates) to position accumulator
- now super-slow speeds are possible and car cannot be stuck on 'digital rails' (xyz)
Have noticed an issue for some time - physics objects resisted starting to move
- seems to be related to 1000Hz update: object velocity not fast enough after 1 frame
- fixed by reducing the speed at which an object is considered to have started moving
Cleaned up code from recent changes / discarded finished notes / browsed old notes
- copied a few things from the old notes onto a short list of tasks to do for release
- although we are still a way from there I want to know what remains other than tyres
- random OT: made arrow keys work to move between options screens (seems like should)
- old notes reminded me that smoke particles do not pick up lighting from lightmap
- the lightmap stores info about artificial light and sky visibility, in an octree
- that means smoke looks black in the night and also too bright in tunnel during day
- made smoke particles read lightmap - see the attached images for before and after
Hello everyone.
As you know, we are working hard and trying to get to the long awaited release. I've discussed a little with Eric about the need for us to try to bring our work to a close and really get to the release.
There's always a problem of wanting to do more but sometimes we need to wrap up what is done and get it out there.
So that's the aim. Currently I am trying to get the last notes off my lists of things to finish and bugs to fix, related to the 1000 Hz updates, multithreading and the lighting changes.
That will leave me with the tyre physics to dive into once again to figure out where it's strong, where it's weak, what parts need to be improved before the big release and what can be done later.
On Eric's side, I know he has been very involved with South City and its architecture, making a really beautiful area with so many ways to drive around. I guess he he will be able to delay some of the planned updates, to focus more on filling holes and there are also holes to fill at Kyoto.
In case you haven't seen already, there are a lot of pictures of South City and other new tracks on the 20th Anniversary report from August:
https://www.lfs.net/20th-anniversary
If you want to know what I have been doing, I have listed it in extreme detail on recent threads.
Multithreading progress: (not adding to this one)
https://www.lfs.net/forum/thread/100464-Progress-on-Multithreading
Development progress: (still ongoing)
https://www.lfs.net/forum/thread/101348-Development-Progress
So I think we are keeping you pretty well informed. I'm sure everyone can understand that it is in no way possible to finish everything before Christmas and we can't really think of a way to produce a new progress report at the moment.
But we are trying to get there as the release is very important!
I will close this thread as it doesn't really serve a useful purpose any more.
As you know, we are working hard and trying to get to the long awaited release. I've discussed a little with Eric about the need for us to try to bring our work to a close and really get to the release.
There's always a problem of wanting to do more but sometimes we need to wrap up what is done and get it out there.
So that's the aim. Currently I am trying to get the last notes off my lists of things to finish and bugs to fix, related to the 1000 Hz updates, multithreading and the lighting changes.
That will leave me with the tyre physics to dive into once again to figure out where it's strong, where it's weak, what parts need to be improved before the big release and what can be done later.
On Eric's side, I know he has been very involved with South City and its architecture, making a really beautiful area with so many ways to drive around. I guess he he will be able to delay some of the planned updates, to focus more on filling holes and there are also holes to fill at Kyoto.
In case you haven't seen already, there are a lot of pictures of South City and other new tracks on the 20th Anniversary report from August:
https://www.lfs.net/20th-anniversary
If you want to know what I have been doing, I have listed it in extreme detail on recent threads.
Multithreading progress: (not adding to this one)
https://www.lfs.net/forum/thread/100464-Progress-on-Multithreading
Development progress: (still ongoing)
https://www.lfs.net/forum/thread/101348-Development-Progress
So I think we are keeping you pretty well informed. I'm sure everyone can understand that it is in no way possible to finish everything before Christmas and we can't really think of a way to produce a new progress report at the moment.
But we are trying to get there as the release is very important!
I will close this thread as it doesn't really serve a useful purpose any more.
Eleven days since the last report, here is some more progress. Not much on multithreading but a couple of thread-related bug fixes. This period started with some work on the skin downloads which now work more smoothly and with a smaller glitch when the texture is applied to the car.
More interesting was an increase in the number of available objects. I had to overcome a technical limitation that object indices were 16-bit numbers, for historical reasons, which meant there was a hard limit of 65536 objects in the world. Objects mainly include actual objects, track cross-sections (we call them simply 'sections') and the segments that link sections, with curves and surfaces. Also flags, marshals, trees, and abstract objects like cameras, start points, grids and pit zones count as objects too.
The new South City has been getting close to that limit and I wanted to add a few more, which were related to an improved way of timing laps. Now that timing can be done to an accuracy of 0.001 seconds, the old path-based timing really wasn't up to the task. The path nodes don't align perfectly with the visual finish line which could give some strange results if there was a photo finish. To be fair these new checkpoints would only add 20 or so objects to a track but when I mentioned them to Eric he did talk about the shortage of object slots and that's why I decided to get down and sort that out. Actually it turned out not to be too bad, as in recent years the obstacles preventing me from changing to 32-bit object indices have been removed. Primarily it was the old 'optimiser' data that switched on and off objects as you drove around the track. It was huge data but now we use an occlusion octree that works in a different way and does not store the indices of individual objects.
There is more explanation in the log below. To illustrate the checkpoints update I have attached 3 screenshots. Because timing no longer relies on paths, it is interesting to note that open configs can now naturally do lap timing even without custom checkpoints being added. Paths still have some advantages such as instantly updated race position list and the colour coding for other cars on the small map, and these functions still work on configurations that do have a path.
Log of changes:
Skin downloads are now converted to DDS and saved from the download thread
- this reduces a graphical glitch when a skin is loaded onto a remote car
- simplified some texture loading code that was mixed up with skin saving
Cleaned up more code around texture loading and skin downloading
- the car's model is no longer rebuilt after its skin is downloaded
- instead the car holds a placeholder skin which is replaced behind the scenes
Support for ALPHA in section ends (tracks are made of segments between cross sections)
Fixed crash bug in the unused texture removal system (could remove some still in use)
Noticed a thread-related bug to do with setting the exposure while restarting
- game code does an 'illegal' D3D call that can cause corruption
Separated "set up lighting" into physical and graphical sections to fix the bug
- considered that this section of code has not yet been separated properly
- the physical part should be done every physics frame (sun could affect physics)
- graphical part needs to be done every graphical frame (shader constants, etc)
Realised that Eric was spammed with too many code-related debug messages in editor
- added a debug level option so he can see more relevant but not excessive messages
- used special macros to simplify the adding of messages for 3 different debug levels
- searched for and updated around 600 debug messages to use the new switchable system
- removed obsolete messages, e.g. a config without a defined path is valid these days
Encountered a thread-related crash, again during restart (clash with drawing humans)
Fixed a zoom bug in one type of trackside TV camera (probably was there a long time)
Removed another thread warning message that appeared in track editor (on undo or load)
Made car ambient shadows, lightmap lighting and headlight effect work in track editor
Considered a new timing system that uses proper checkpoints like in the layout editor
- existing timing is based on path nodes and is misaligned with visible finish lines
- Eric mentioned a concern with the number of objects in map approaching a hard limit
Converted object indices to 32-bit to increase the number of objects that can be added
- previous 16-bit indices only allowed a maximum of 65536 objects to be added to a map
- number includes objects, buildings, cameras, track cross sections and segments, etc.
- in the past the occlusion data relied on object indices and took up a lot of memory
- now we have a totally different occlusion system that storage issue no longer exists
- it took only a bit more than a day to update the remaining object indices to 32-bit
Moved onto the new system that allows accurate alignment of checkpoints and finish line
- started by analysing all path nodes over marked track segments to detect checkpoints
- this produces many checkpoints, often duplicates, e.g. a finish line for every config
- the duplicate checkpoints are combined and config switches enable them appropriately
- this leaves fully working checkpoints that Eric can adjust to improve accuracy
- worked on the editor side of it (the checkpoints are a type of invisible object)
- on the driving side, checkpoint detection use existing code for layout checkpoints
- timing is no longer path-based but paths have uses, e.g. instant race position list
- interestingly open configs can now do timing even without adding custom checkpoints
More interesting was an increase in the number of available objects. I had to overcome a technical limitation that object indices were 16-bit numbers, for historical reasons, which meant there was a hard limit of 65536 objects in the world. Objects mainly include actual objects, track cross-sections (we call them simply 'sections') and the segments that link sections, with curves and surfaces. Also flags, marshals, trees, and abstract objects like cameras, start points, grids and pit zones count as objects too.
The new South City has been getting close to that limit and I wanted to add a few more, which were related to an improved way of timing laps. Now that timing can be done to an accuracy of 0.001 seconds, the old path-based timing really wasn't up to the task. The path nodes don't align perfectly with the visual finish line which could give some strange results if there was a photo finish. To be fair these new checkpoints would only add 20 or so objects to a track but when I mentioned them to Eric he did talk about the shortage of object slots and that's why I decided to get down and sort that out. Actually it turned out not to be too bad, as in recent years the obstacles preventing me from changing to 32-bit object indices have been removed. Primarily it was the old 'optimiser' data that switched on and off objects as you drove around the track. It was huge data but now we use an occlusion octree that works in a different way and does not store the indices of individual objects.
There is more explanation in the log below. To illustrate the checkpoints update I have attached 3 screenshots. Because timing no longer relies on paths, it is interesting to note that open configs can now naturally do lap timing even without custom checkpoints being added. Paths still have some advantages such as instantly updated race position list and the colour coding for other cars on the small map, and these functions still work on configurations that do have a path.
Log of changes:
Skin downloads are now converted to DDS and saved from the download thread
- this reduces a graphical glitch when a skin is loaded onto a remote car
- simplified some texture loading code that was mixed up with skin saving
Cleaned up more code around texture loading and skin downloading
- the car's model is no longer rebuilt after its skin is downloaded
- instead the car holds a placeholder skin which is replaced behind the scenes
Support for ALPHA in section ends (tracks are made of segments between cross sections)
Fixed crash bug in the unused texture removal system (could remove some still in use)
Noticed a thread-related bug to do with setting the exposure while restarting
- game code does an 'illegal' D3D call that can cause corruption
Separated "set up lighting" into physical and graphical sections to fix the bug
- considered that this section of code has not yet been separated properly
- the physical part should be done every physics frame (sun could affect physics)
- graphical part needs to be done every graphical frame (shader constants, etc)
Realised that Eric was spammed with too many code-related debug messages in editor
- added a debug level option so he can see more relevant but not excessive messages
- used special macros to simplify the adding of messages for 3 different debug levels
- searched for and updated around 600 debug messages to use the new switchable system
- removed obsolete messages, e.g. a config without a defined path is valid these days
Encountered a thread-related crash, again during restart (clash with drawing humans)
Fixed a zoom bug in one type of trackside TV camera (probably was there a long time)
Removed another thread warning message that appeared in track editor (on undo or load)
Made car ambient shadows, lightmap lighting and headlight effect work in track editor
Considered a new timing system that uses proper checkpoints like in the layout editor
- existing timing is based on path nodes and is misaligned with visible finish lines
- Eric mentioned a concern with the number of objects in map approaching a hard limit
Converted object indices to 32-bit to increase the number of objects that can be added
- previous 16-bit indices only allowed a maximum of 65536 objects to be added to a map
- number includes objects, buildings, cameras, track cross sections and segments, etc.
- in the past the occlusion data relied on object indices and took up a lot of memory
- now we have a totally different occlusion system that storage issue no longer exists
- it took only a bit more than a day to update the remaining object indices to 32-bit
Moved onto the new system that allows accurate alignment of checkpoints and finish line
- started by analysing all path nodes over marked track segments to detect checkpoints
- this produces many checkpoints, often duplicates, e.g. a finish line for every config
- the duplicate checkpoints are combined and config switches enable them appropriately
- this leaves fully working checkpoints that Eric can adjust to improve accuracy
- worked on the editor side of it (the checkpoints are a type of invisible object)
- on the driving side, checkpoint detection use existing code for layout checkpoints
- timing is no longer path-based but paths have uses, e.g. instant race position list
- interestingly open configs can now do timing even without adding custom checkpoints
Six more days have elapsed and there is a little more progress to report. It's interesting that most of the work was really nothing to do with multithreading. I had to remove a system that used to clear out disused skins from RAM, because the way it worked was not thread-safe. As mentioned before, we needed a new system that works with all car textures (not just skins). So I've implemented that as described below. You can see it doing its thing in the attached screenshot.
Another milestone is that I sent a version to Eric with a cautious recommendation that he could try using it for development. It has been stable apart from one thread-related crash - an FF device was being uninitialised at the same time as it was being used on another thread.
One of the next steps is a pit-out glitch reduction system that is a lot better than the old one. To reduce a glitch when a remote player leaves the pits in the current version of LFS, while you are driving, no textures are loaded from HDD (or SSD, obviously). The car model is built but may appear white where there should be textures, if those textures were not already loaded for some other car. It has often been reported as a "white cars" bug although it's really not a bug at all, and their cars are rebuilt with textures when you stop at some point.
In the new system (as reported on Sat 29 Oct) a remote player's car model is not built as soon as they leave the pits, but only when they are within your view. But currently all their textures will be loaded immediately as I had to remove the white cars pit-out glitch reduction system.
My new plan is to make it that a car may appear white for a short time, but one by one over the coming frames, the textures will be loaded. The entire model will not have to be rebuilt at any later point, only the actual textures will be loaded, without the car's model even noticing that. This will also cover a similar situation when a remote player joins and their skin is downloaded. That skin will be applied to their car without needing a full rebuild. I expect this to take a few days. The principle is that when a texture is requested, if it is not already in RAM, all information that is required to load it is stored in a "Texture" object that doesn't have its own D3D texture (instead pointing to a small plain system texture, probably grey). The real D3D texture will be loaded at some point (up to a few a few seconds later) and replaced in the Texture object.
Now that multithreading is basically done, I'll stop reporting daily progress and go back to a more normal style. I hope to report more regularly than in the past, as I know the reports are interesting for some people and the use of a closed thread has allowed me to make unofficial reports without them becoming a distraction.
I hope the detailed daily calendar has been of some interest, and some insight into the general complication of the work done by a game programmer. We don't just do some super-slow magic, conjuring up cars and tracks. Instead, it's mostly a lot of strange and abstract stuff that you might not think of immediately. It has been interesting for me, looking back and seeing all this stuff and helps me to understand why my time estimates are always so far off.
Anyway, here's the last of the daily detail calendars, for now:
31 Oct (evening)
FIX: Escape from MPR click to point in replay continued visibly speeding after Esc
FIX: Exposure was not reset after exiting from game which could cause overexposure
1 Nov
FIX: Lights were not switched on when starting a night replay after daylight replay
Removed function "check_for_skin_space" which was accessing Race at the wrong time
- this function ran through texture and car lists to remove older skins from memory
- it is not thread-safe as it was accessing Race (game code) during graphical render
- a better way to identify unused textures (not only skins) is now needed due to mods
Gathered remaining things to do onto new sheets of papers to make them more readable
FIX: Crash when loading very old cars, then a memory leak when loading very old cars
Sent update to Eric with suggestion that it may now be safe to use for development
2 Nov
As mentioned we need a system to remove textures and skins from cars no longer in use
Started coding a method to detect which already loaded textures can be deleted from RAM
First step: to know exactly which textures are used by cars (and helmets) but not world
- imperative not to delete any textures that are still in use or there will be a crash
- change all texture and material initialisation functions to specify usage (car/world)
- this also applies to "re-use" functions, in case a car texture is later used by world
Eric reported a crash bug with ALT+TAB from full screen and provided the crash address
- crash address was in LFS code, setting a Force (on a FFDevice that still existed)
- thread-related: controllers were being shut down while game code applied a force
3 Nov
Examined yesterday's code and brushed up a bit, ready to start with the second step
The algorithm must identify all car textures that are not in use and can be deleted
- avoids an excessive build-up of textures in RAM as cars come and go over time
- run through all textures marked CAR and IN USE - set to DISUSED (and add time stamp)
- new 3D object and car functions to run through all textures and mark then as IN USE
- call car functions on every car in the race, game setup screen, garage (and helmet)
- whatever remains as DISUSED is no longer used by an existing car so can be deleted
Now the focus is on when to run the described algorithm and which textures to delete
- we don't want to delete textures every time it's possible as they may be needed soon
- for example a player who goes to the pits is likely to exit again with the same car
- another example: whole game exits to game setup screen - no point removing textures
- the "last used" time stamp could help to delete textures not in use for a while
Wrote a system to remove materials and textures after some time
- if any car is added or removed in game, a short countdown is started
- after 3 seconds, disused materials and textures are identified (algorithm above)
- then 1 disused material or 1 disused texture is deleted every graphical frame
- but only materials and textures set to unused at least 5 minutes ago are deleted
4 Nov (morning)
Yesterday's algorithm does the job and unused textures always get removed eventually
- avoids build-up in RAM but will sometimes remove textures that will be needed again
- algorithm can be adjusted later but now it's best to work on delayed texture loading
- cleaned up some code and added debug check for texture delete while material exists
Another milestone is that I sent a version to Eric with a cautious recommendation that he could try using it for development. It has been stable apart from one thread-related crash - an FF device was being uninitialised at the same time as it was being used on another thread.
One of the next steps is a pit-out glitch reduction system that is a lot better than the old one. To reduce a glitch when a remote player leaves the pits in the current version of LFS, while you are driving, no textures are loaded from HDD (or SSD, obviously
). The car model is built but may appear white where there should be textures, if those textures were not already loaded for some other car. It has often been reported as a "white cars" bug although it's really not a bug at all, and their cars are rebuilt with textures when you stop at some point.In the new system (as reported on Sat 29 Oct) a remote player's car model is not built as soon as they leave the pits, but only when they are within your view. But currently all their textures will be loaded immediately as I had to remove the white cars pit-out glitch reduction system.
My new plan is to make it that a car may appear white for a short time, but one by one over the coming frames, the textures will be loaded. The entire model will not have to be rebuilt at any later point, only the actual textures will be loaded, without the car's model even noticing that. This will also cover a similar situation when a remote player joins and their skin is downloaded. That skin will be applied to their car without needing a full rebuild. I expect this to take a few days. The principle is that when a texture is requested, if it is not already in RAM, all information that is required to load it is stored in a "Texture" object that doesn't have its own D3D texture (instead pointing to a small plain system texture, probably grey). The real D3D texture will be loaded at some point (up to a few a few seconds later) and replaced in the Texture object.
Now that multithreading is basically done, I'll stop reporting daily progress and go back to a more normal style. I hope to report more regularly than in the past, as I know the reports are interesting for some people and the use of a closed thread has allowed me to make unofficial reports without them becoming a distraction.
I hope the detailed daily calendar has been of some interest, and some insight into the general complication of the work done by a game programmer. We don't just do some super-slow magic, conjuring up cars and tracks. Instead, it's mostly a lot of strange and abstract stuff that you might not think of immediately. It has been interesting for me, looking back and seeing all this stuff and helps me to understand why my time estimates are always so far off.
Anyway, here's the last of the daily detail calendars, for now:
31 Oct (evening)
FIX: Escape from MPR click to point in replay continued visibly speeding after Esc
FIX: Exposure was not reset after exiting from game which could cause overexposure
1 Nov
FIX: Lights were not switched on when starting a night replay after daylight replay
Removed function "check_for_skin_space" which was accessing Race at the wrong time
- this function ran through texture and car lists to remove older skins from memory
- it is not thread-safe as it was accessing Race (game code) during graphical render
- a better way to identify unused textures (not only skins) is now needed due to mods
Gathered remaining things to do onto new sheets of papers to make them more readable
FIX: Crash when loading very old cars, then a memory leak when loading very old cars
Sent update to Eric with suggestion that it may now be safe to use for development
2 Nov
As mentioned we need a system to remove textures and skins from cars no longer in use
Started coding a method to detect which already loaded textures can be deleted from RAM
First step: to know exactly which textures are used by cars (and helmets) but not world
- imperative not to delete any textures that are still in use or there will be a crash
- change all texture and material initialisation functions to specify usage (car/world)
- this also applies to "re-use" functions, in case a car texture is later used by world
Eric reported a crash bug with ALT+TAB from full screen and provided the crash address
- crash address was in LFS code, setting a Force (on a FFDevice that still existed)
- thread-related: controllers were being shut down while game code applied a force
3 Nov
Examined yesterday's code and brushed up a bit, ready to start with the second step
The algorithm must identify all car textures that are not in use and can be deleted
- avoids an excessive build-up of textures in RAM as cars come and go over time
- run through all textures marked CAR and IN USE - set to DISUSED (and add time stamp)
- new 3D object and car functions to run through all textures and mark then as IN USE
- call car functions on every car in the race, game setup screen, garage (and helmet)
- whatever remains as DISUSED is no longer used by an existing car so can be deleted
Now the focus is on when to run the described algorithm and which textures to delete
- we don't want to delete textures every time it's possible as they may be needed soon
- for example a player who goes to the pits is likely to exit again with the same car
- another example: whole game exits to game setup screen - no point removing textures
- the "last used" time stamp could help to delete textures not in use for a while
Wrote a system to remove materials and textures after some time
- if any car is added or removed in game, a short countdown is started
- after 3 seconds, disused materials and textures are identified (algorithm above)
- then 1 disused material or 1 disused texture is deleted every graphical frame
- but only materials and textures set to unused at least 5 minutes ago are deleted
4 Nov (morning)
Yesterday's algorithm does the job and unused textures always get removed eventually
- avoids build-up in RAM but will sometimes remove textures that will be needed again
- algorithm can be adjusted later but now it's best to work on delayed texture loading
- cleaned up some code and added debug check for texture delete while material exists
Three days later and I've reached the end of a shader update so thought I'd post another of these lists and a few images. Sometimes when updating graphics code I need to be careful not to break something, and that is surely the case when changing the way shaders work. So I produce screenshots from exact locations as I go, to do direct comparison by flicking between them. I realised some more of the shaders should be combined so now 24 shadowable, in game pixel shaders have been narrowed down to 9 with a few options. Actually there are two versions of each (with and without use of shadow maps) so it's really 48 down to 18.
On a day where I was feeling a bit interested in some other things I'd like to try, I felt uneasy with the number of things still on lists so made myself grind through a few fixes on paper and TODO comments in the code. I felt a bit uninspired but got more motivated as I began to cross things off lists.
During that time, another shader bugged me a bit, as there was a certain inconsistency in the shaders. Previously just one type of alpha shader had been a special case. The shiny windows shader (for e.g. the marshall boxes at Rockingham, and now several South City buildings) had previously been the only one that needed a different alpha blending state, where the source output from the pixel shader is not multiplied by the alpha value. Otherwise because such a transparent material (glass) would have a low alpha value that would remove the shine on the window, which is nearly all you can see. So in that shader the source reduction is done within the shader. But that one special case was no longer the only one, as the new shader I reported in the previous update also needed that change. But the remaining shaders didn't need to be how they were and could also move to the same method. Through a series of updates I've been able to make that the same in all the alpha shaders, so now there are no changes of blend state when drawing the alpha pass. Well I guess there are, if headlights are shining on things, but I'm talking about the majority of objects. So that cleans up the code a bit and removes hundreds of state changes.
Again, it's not a big change but I sometimes need to take a bit of time to clean up the code after some new things are added over the years, otherwise it becomes unmanageable. In this case there was a slight change to the output of other of the shaders, which was due to a change that I interpret as a 'bug' in the old system. But as it caused a slight change to the shine on transparent objects, I had to confirm that with Eric.
So I hope that's a little more insight into the way things go. It's not really related to multithreading but I don't see any reason to be on shaders next week.
The attached images show the reduction in calls to SetBlendState, through first an increase, then a bug, then the final result. They are described in the "22 Oct" paragraph.
Here's the log of updates:
20 Oct
Some other shaders with only tiny differences came to mind, decided to combine them
- now the shadowable, main world pixel shaders have been reduced from 24 to 9 total
- also there were was some more cleaning up and documentation needed in the shaders
21 Oct
Would like to try some ideas but must finish and fix a few things even if boring
Did some TODOs and fixed a bug in editor 'new car' related to storing setup in car
Spotted and fixed a bug in game setup screen that caused car to constantly reload
Fixed an already noted bug causing single player to superspeed a while after unpause
Updated exit from path editor that referenced Race - now use new style exit function
Fixed crash in game after exiting from animation editor if any AI drivers on track
Something was still bugging me about one of the shaders which seems inconsistent
- the variable shine / alpha shader effectively multiplies diffuse by alpha twice
- possible to correct this with a few changes, including a change to alpha blend state
- struggled a while with an 'impossible' bug that turned out to be a compiler error :-/
- blend state now consistent with other alpha shaders which used that for other reasons
- there is a slight change to appearance, did comparisons and asked Eric to check if OK
22 Oct
Now that alpha + vshine shader is consistent with others, improvements are possible
- there should be no need to change the alpha blending state during the alpha passes
- as seen in recent screenshots, counters can display frame state changes in real time
- D3D11 does render state changes in blocks instead of individual states as in D3D9
- state counters 'SetRenderState' and 'SetSamplerState' were left from D3D9 version
- removed them and added SetBlendState, SetRasterState, SetDepthState, SetSamplerState
- screenshots show original number of states then 1st update *increased* SetBlendState
- that is because one shader, used for trees, SrcBlend is still D3D11_BLEND_SRC_ALPHA
- 3rd image now all alpha shaders updated, so no calls to SetBlendState in alpha pass
- but there is a bug, see the distant trees, fog appears on their transparent parts
- that is due to the alpha blending change, now fixed in shader - 4th image all good
- clean up: some state requests were no longer needed, now constant during alpha pass
On a day where I was feeling a bit interested in some other things I'd like to try, I felt uneasy with the number of things still on lists so made myself grind through a few fixes on paper and TODO comments in the code. I felt a bit uninspired but got more motivated as I began to cross things off lists.
During that time, another shader bugged me a bit, as there was a certain inconsistency in the shaders. Previously just one type of alpha shader had been a special case. The shiny windows shader (for e.g. the marshall boxes at Rockingham, and now several South City buildings) had previously been the only one that needed a different alpha blending state, where the source output from the pixel shader is not multiplied by the alpha value. Otherwise because such a transparent material (glass) would have a low alpha value that would remove the shine on the window, which is nearly all you can see. So in that shader the source reduction is done within the shader. But that one special case was no longer the only one, as the new shader I reported in the previous update also needed that change. But the remaining shaders didn't need to be how they were and could also move to the same method. Through a series of updates I've been able to make that the same in all the alpha shaders, so now there are no changes of blend state when drawing the alpha pass. Well I guess there are, if headlights are shining on things, but I'm talking about the majority of objects. So that cleans up the code a bit and removes hundreds of state changes.
Again, it's not a big change but I sometimes need to take a bit of time to clean up the code after some new things are added over the years, otherwise it becomes unmanageable. In this case there was a slight change to the output of other of the shaders, which was due to a change that I interpret as a 'bug' in the old system. But as it caused a slight change to the shine on transparent objects, I had to confirm that with Eric.
So I hope that's a little more insight into the way things go. It's not really related to multithreading but I don't see any reason to be on shaders next week.
The attached images show the reduction in calls to SetBlendState, through first an increase, then a bug, then the final result. They are described in the "22 Oct" paragraph.
Here's the log of updates:
20 Oct
Some other shaders with only tiny differences came to mind, decided to combine them
- now the shadowable, main world pixel shaders have been reduced from 24 to 9 total
- also there were was some more cleaning up and documentation needed in the shaders
21 Oct
Would like to try some ideas but must finish and fix a few things even if boring
Did some TODOs and fixed a bug in editor 'new car' related to storing setup in car
Spotted and fixed a bug in game setup screen that caused car to constantly reload
Fixed an already noted bug causing single player to superspeed a while after unpause
Updated exit from path editor that referenced Race - now use new style exit function
Fixed crash in game after exiting from animation editor if any AI drivers on track
Something was still bugging me about one of the shaders which seems inconsistent
- the variable shine / alpha shader effectively multiplies diffuse by alpha twice
- possible to correct this with a few changes, including a change to alpha blend state
- struggled a while with an 'impossible' bug that turned out to be a compiler error :-/
- blend state now consistent with other alpha shaders which used that for other reasons
- there is a slight change to appearance, did comparisons and asked Eric to check if OK
22 Oct
Now that alpha + vshine shader is consistent with others, improvements are possible
- there should be no need to change the alpha blending state during the alpha passes
- as seen in recent screenshots, counters can display frame state changes in real time
- D3D11 does render state changes in blocks instead of individual states as in D3D9
- state counters 'SetRenderState' and 'SetSamplerState' were left from D3D9 version
- removed them and added SetBlendState, SetRasterState, SetDepthState, SetSamplerState
- screenshots show original number of states then 1st update *increased* SetBlendState
- that is because one shader, used for trees, SrcBlend is still D3D11_BLEND_SRC_ALPHA
- 3rd image now all alpha shaders updated, so no calls to SetBlendState in alpha pass
- but there is a bug, see the distant trees, fog appears on their transparent parts
- that is due to the alpha blending change, now fixed in shader - 4th image all good
- clean up: some state requests were no longer needed, now constant during alpha pass
Thanks for the link! I watched it with the family yesterday evening. Interesting to see how Peter has changed. Well, I remember I was in my late 20s when we all went to his 40th birthday party, and now I'm 50, so he must be in his 60s now.
Nice that Eric and I were kindly mentioned.
That story about how we met isn't quite true though it's based on facts. That's kind of what Peter does.
In fact I did work as a motorbike courier for a company called Black & White that was based in Twickenham. But I was looking for a job as a programmer (I had previously worked at a company called Digital Integration, on the Hind helicopter simulator) so I contacted a recruitment agency that specialised in game jobs. Quite quickly they came up with this interview, with someone who I would surely admire (Peter Molyneux). Actually I hadn't even heard of him before but went for the interview as it sounded good. I rode down on my bike and had a good meeting there, in which we of course talked about my job at Black & White, which was funny as the interview was about working on the game called Black & White.
I had a nice demo (with an animated car and some people and dogs that could walk around on a lumpy landscape - actually it's the same program that evolved into LFS) so they were able to offer me the job on the spot. I said, thanks and I'll consider it. But he looked a bit shocked and said, why aren't you accepting the job now? I said, well I thought that it was the right thing to do... and actually I decided to just accept it then anyway. That was the right move as it was a very good job in the end although there were ups and downs.
We got it running a few years ago, using a boxed game and the fan patches you can find online. Leo and Nicole enjoyed it quite a bit.
How about this attached picture from the video, you can see a creature editor I built in game, using my own buttons system that I had imported into Black and White.
Compare that with the attached image of the vehicle editor.
The early LFS in-game menu style was more inspired by Quake 3 Arena.
One week later... where did that time go! 
I've been doing a variety of updates including a bit of off topic into shaders that is finished for now. Some cleaning up and organising. Sometimes I add a line of code in a convenient place. Then something conceptually similar comes along and I add it at the same place. Then that happens again and again and it's taking up a lot of space in the original source file. So it becomes a new system in itself, or at least something that should be packaged in its own function, and might end up with its own source code file. This time that happened for the "sync point" which was starting to get messy right at the end of the main loop. Some other clutter from Main.cpp went along into SyncPoint.cpp too. The Sync Point is a very useful place for LFS to do things, because for that tiny slice of time, only the main thread is running and the game/physics thread is waiting, so it's a safe time for things that need safety. As a random example, suppose the user refreshed controllers. It is done at the sync point, so the controller can't be updated half way through a game update. Anyway that was really just a minor change. Randomly I noticed that I wanted to use SHIFT+J or SHIFT+S sometimes in Garage and Free View but those keys weren't available. So veered off topic a while to sort that out.
I realised there were more ways to check thread-safety and did something for the "Physics" system which manages the interactions between and updates of cars and other moving objects. I gave it a protected access function like I had done for Race and Snapshot before.
Eric noticed an issue with something involving two layers of alpha textures. I ended up solving a newly introduced editor crash and also went full off-topic for a while adding a new shader which is really for a new type of transparent window on buildings that will be useful in a few places. It can use a single pass instead of two passes which is always preferable. But it was a bit confusing at first so I reorganised the shader definitions, to the point where I got a new understanding of them and was able to come up a with a way to reduce the number of shaders. It is to help avoid a phenomenon called the "Shader Permutation Explosion" which is where the number of shaders in use can get out of hand. Well LFS hadn't got to the point of an "explosion" yet but it was bugging me that some shaders were extremely similar and I found a way to use a few options (in the form of shader constants) to produce the different effects instead of increasing the number of shaders. The attached images show the reduction in calls to "SetPixelShader" so that is a slight saving (although I couldn't actually measure a change in frame rate with such small numbers).
Another quick change solved an audio problem and got something off a list of things to do. You can see in the attached image, the audio updates (shown in yellow) occur at approximately 100Hz and are not affected by graphical frame rate. This is the maximum update rate I can get with the current system and results in the most responsive sound, even if the graphical frame rate drops a bit.
From tomorrow I plan to try and get a few more things off lists as I'd like to get to the end of the multithreading subject as soon as possible.
Here's the diary of updates.
13 Oct
Wanted to remove the last remaining old-style reference to Race, in Misc options
- needed another "must do x" in the sync point in the main loop, but getting messy
- added a new source code file for things to do at the sync point, cleaned up a bit
- removed Race from external visibility, now only accessible through protected function
Off topic update: added SHIFT+J and SHIFT+S keys to garage screen and free view mode
- this also involved quite a bit of cleaning up and improving some packet functions
- also added SHIFT+P to free view mode (SHIFT+P/J no check for unsaved layout changes)
14 Oct
Some quick fixes around cars in garage and other screens accessing Race when loading
Went through a list of source code files to check if they accessed anything wrongly
Decided to update the 'Physics' system to use a thread-safe access check like Race
- the Physics system is the code that manages all the moving objects (Cars and Solids)
- like Race it must not be accessed from the main / graphics code at the wrong time
- not a big task in this case but again identified issues and improvements were made
15 Oct
Most of morning with family
Off topic: researched an issue for Eric - modeller object appeared different in game
- after the research my version crashed when exiting from Track Editor using X button
- it turned out this was due to a new 'thread-safe' method of exiting track editor
- but the correct exit function was not called when using the window's X button
- implemented a way to provide a special exit function for any screen, used on TrackEd
- found & fixed a bug exiting from the modeller in TrackEd, related to Map Squares
- continued to track down non-thread-safe code switching into and out of TrackEd
16 Oct
Morning with family
Off topic: tried a new shader to allow something like 'overlay' but transparent
- in current version of LFS an 'overlay' shader is available for black (solid) windows
- went through code to allow overlay with ALPHA materials and added a new shader for it
17 Oct
Fixed a bug reported by Eric and checked another bug that had already been fixed
Off topic: cleaned up the shader defines that control the creation of various shaders
- many of the shaders are from one hlsl file with various sections enabled or disabled
- the organisation of these switches was confusing and could be improved / simplified
- this was done carefully with plenty of testing so as not to break any of the shaders
- it is now a lot easier to understand the similarities and differences between shaders
18 Oct
Bit of leaf clearing from gutters in the morning - house below 3 large beech trees...
Some more renaming and organising of the shader names and specifications in game code
Investigation into possibility of using shader options to reduce number of shaders
- some of the shaders are so similar with just a couple of different multipliers
19 Oct
Change to sound/audio code, sound updates are done at maximum rate of 100 per second
Previous changes resulted in 1 sound update per graphical frame which solved an issue
- the original problem was that sound updates could delay the sync point and graphics
- intermediate solution was to only do sound on the next update after the sync point
- but that was less immediate and caused poor sound quality if the frame rate dropped
- new solution: update whenever possible but delay 1ms if main thread waiting for sync
- you can see the audio updates (yellow) in the attached thread timing graph
- I produce these timing graphs in the debug version to exaggerate the CPU time
- another thing to notice is that every 10th physics update is longer than usual
- this is due to only checking for contact collisions every 0.01 second
- checking against the environment was the main CPU cost of the 1000Hz physics
- so contact points that are not currently touching wait 10 updates before next check
A new shader update combines 6 groups of 3 pixel shaders into 6 shaders total
- the idea is to reduce the number of extremely similar shaders in memory
- changing from one shader to another involves a tiny time penalty so good to reduce
- the new version uses shader constants to create the different effects instead
- these minor differences were about shine level and diffuse colour depending on alpha
- as you can see in the attached images there are now fewer calls to "SetPixelShader"
- it's not a big change but means the new shader has not resulted in even more shaders
I've been doing a variety of updates including a bit of off topic into shaders that is finished for now. Some cleaning up and organising. Sometimes I add a line of code in a convenient place. Then something conceptually similar comes along and I add it at the same place. Then that happens again and again and it's taking up a lot of space in the original source file. So it becomes a new system in itself, or at least something that should be packaged in its own function, and might end up with its own source code file. This time that happened for the "sync point" which was starting to get messy right at the end of the main loop. Some other clutter from Main.cpp went along into SyncPoint.cpp too. The Sync Point is a very useful place for LFS to do things, because for that tiny slice of time, only the main thread is running and the game/physics thread is waiting, so it's a safe time for things that need safety. As a random example, suppose the user refreshed controllers. It is done at the sync point, so the controller can't be updated half way through a game update. Anyway that was really just a minor change. Randomly I noticed that I wanted to use SHIFT+J or SHIFT+S sometimes in Garage and Free View but those keys weren't available. So veered off topic a while to sort that out.
I realised there were more ways to check thread-safety and did something for the "Physics" system which manages the interactions between and updates of cars and other moving objects. I gave it a protected access function like I had done for Race and Snapshot before.
Eric noticed an issue with something involving two layers of alpha textures. I ended up solving a newly introduced editor crash and also went full off-topic for a while adding a new shader which is really for a new type of transparent window on buildings that will be useful in a few places. It can use a single pass instead of two passes which is always preferable. But it was a bit confusing at first so I reorganised the shader definitions, to the point where I got a new understanding of them and was able to come up a with a way to reduce the number of shaders. It is to help avoid a phenomenon called the "Shader Permutation Explosion" which is where the number of shaders in use can get out of hand. Well LFS hadn't got to the point of an "explosion" yet but it was bugging me that some shaders were extremely similar and I found a way to use a few options (in the form of shader constants) to produce the different effects instead of increasing the number of shaders. The attached images show the reduction in calls to "SetPixelShader" so that is a slight saving (although I couldn't actually measure a change in frame rate with such small numbers).
Another quick change solved an audio problem and got something off a list of things to do. You can see in the attached image, the audio updates (shown in yellow) occur at approximately 100Hz and are not affected by graphical frame rate. This is the maximum update rate I can get with the current system and results in the most responsive sound, even if the graphical frame rate drops a bit.
From tomorrow I plan to try and get a few more things off lists as I'd like to get to the end of the multithreading subject as soon as possible.
Here's the diary of updates.
13 Oct
Wanted to remove the last remaining old-style reference to Race, in Misc options
- needed another "must do x" in the sync point in the main loop, but getting messy
- added a new source code file for things to do at the sync point, cleaned up a bit
- removed Race from external visibility, now only accessible through protected function
Off topic update: added SHIFT+J and SHIFT+S keys to garage screen and free view mode
- this also involved quite a bit of cleaning up and improving some packet functions
- also added SHIFT+P to free view mode (SHIFT+P/J no check for unsaved layout changes)
14 Oct
Some quick fixes around cars in garage and other screens accessing Race when loading
Went through a list of source code files to check if they accessed anything wrongly
Decided to update the 'Physics' system to use a thread-safe access check like Race
- the Physics system is the code that manages all the moving objects (Cars and Solids)
- like Race it must not be accessed from the main / graphics code at the wrong time
- not a big task in this case but again identified issues and improvements were made
15 Oct
Most of morning with family
Off topic: researched an issue for Eric - modeller object appeared different in game
- after the research my version crashed when exiting from Track Editor using X button
- it turned out this was due to a new 'thread-safe' method of exiting track editor
- but the correct exit function was not called when using the window's X button
- implemented a way to provide a special exit function for any screen, used on TrackEd
- found & fixed a bug exiting from the modeller in TrackEd, related to Map Squares
- continued to track down non-thread-safe code switching into and out of TrackEd
16 Oct
Morning with family
Off topic: tried a new shader to allow something like 'overlay' but transparent
- in current version of LFS an 'overlay' shader is available for black (solid) windows
- went through code to allow overlay with ALPHA materials and added a new shader for it
17 Oct
Fixed a bug reported by Eric and checked another bug that had already been fixed
Off topic: cleaned up the shader defines that control the creation of various shaders
- many of the shaders are from one hlsl file with various sections enabled or disabled
- the organisation of these switches was confusing and could be improved / simplified
- this was done carefully with plenty of testing so as not to break any of the shaders
- it is now a lot easier to understand the similarities and differences between shaders
18 Oct
Bit of leaf clearing from gutters in the morning - house below 3 large beech trees...
Some more renaming and organising of the shader names and specifications in game code
Investigation into possibility of using shader options to reduce number of shaders
- some of the shaders are so similar with just a couple of different multipliers
19 Oct
Change to sound/audio code, sound updates are done at maximum rate of 100 per second
Previous changes resulted in 1 sound update per graphical frame which solved an issue
- the original problem was that sound updates could delay the sync point and graphics
- intermediate solution was to only do sound on the next update after the sync point
- but that was less immediate and caused poor sound quality if the frame rate dropped
- new solution: update whenever possible but delay 1ms if main thread waiting for sync
- you can see the audio updates (yellow) in the attached thread timing graph
- I produce these timing graphs in the debug version to exaggerate the CPU time
- another thing to notice is that every 10th physics update is longer than usual
- this is due to only checking for contact collisions every 0.01 second
- checking against the environment was the main CPU cost of the 1000Hz physics
- so contact points that are not currently touching wait 10 updates before next check
A new shader update combines 6 groups of 3 pixel shaders into 6 shaders total
- the idea is to reduce the number of extremely similar shaders in memory
- changing from one shader to another involves a tiny time penalty so good to reduce
- the new version uses shader constants to create the different effects instead
- these minor differences were about shine level and diffuse colour depending on alpha
- as you can see in the attached images there are now fewer calls to "SetPixelShader"
- it's not a big change but means the new shader has not resulted in even more shaders
Last edited by Scawen, .
Again you are just trying to find ways to steal models from other developers and bypass our legality checks.
This "retopo checking system" you mention, is an unpaid volunteer who gets that mod that looks like it comes from another game and does a detailed check, and doesn't receive any pay for it. Maybe you imagine some magical system where a computer somehow does this automatically?
You are treating us as if we are a huge company and you pay a monthly subscription, and we have a team of reviewers on a good salary, defending us from all the annoying customers. But that is not how we operate, and it's very sad that some people like to make an "us and them" type of situation.
You know we are a very tiny team, and because of that we have been able to change a single fee for a lifetime subscription? In my mind I'm part of the community. Yes of course I have a special position here but Eric and I gave up guaranteed high pay and corporate lifestyle over 20 years ago, to work on a project at our own pace. It does make me quite sad that some community members treat us, and the reviewers, like the enemy.
We said, something like "here is a new mods system - please make your own creations or work from legitimate sources and don't rip models from other games". It seems a fair request and we expect mod creators to work with us on that, not fight against it. Now there are people who are constantly trying to mess that up and break the trust and friendship. It's at the point now where this is often a major distraction from actual development. I had a nice day out yesterday to go on a bike ride with the family (last week of school holiday). I dreaded opening this thread to catch up.
Many creators have made super mods, either built from scratch or from legitimate sources. They have made cars inspired by one particular car or multiple cars, and worked to make them look good and be fun to drive. It's so great to see this and eventually give them approved status (again, with the help of community ratings). There is so much freedom to make great mods and it is really heartwarming to see that happening as intended. Let's keep working together to do that.
Yes, Eric sent me the latest update and I've generated a bitmap for new SO. https://www.lfs.net/static/man/SO_new.png
It has the same scale as the bitmaps of the public tracks (1 pixel = 1m).
The bitmaps are generated at 2560x2560. Small parts of driveable road are missing off the top and bottom right of the image.
EDIT: there is another download below with all the driveable roads visible
Last edited by Scawen, .
Nice!
Yes, good idea.
Eric has sent me a daytime version of the same South City view (attached).
20th Anniversary!
Hello Racers,
It is the 20th anniversary of Live for Speed becoming well known! According to various sources, it was on 18th August 2002 that Demo Test 0.04k first became known to more than a few people. It was very exciting to see the reaction to this new racing simulator that seemed to have appeared from nowhere and was quite different to what was available at the time. It had a focus on multiplayer racing. Despite synchronisation issues, it was already a lot of fun to race around Blackwood, with up to 8 drivers, in an XF GTI, XR GT, or XR GT TURBO.
If you'd like to try 0.04k it is available on our downloads archive page. It does not connect to the master server but it is possible to connect to a host directly by IP address.
LFS has come a long way since 2002, with 9 track areas now available and a huge variety of cars created by racers using the new mods system. Of course, there is more to come, notably a new graphics system and new tyre physics. The new graphics system includes dynamic lighting and day to night transitions. The tyre physics update is more realistic and accurate for a better driving experience.
We decided to show a few pictures of some track updates that Eric has been working on and share some technical details about the ongoing development in the physics system.
Please visit the 20th Anniversary page to see the screenshots, read about the development and watch a few short video clips.
Thank you for all the support over the last 20 years!
- LFS Developers
It is the 20th anniversary of Live for Speed becoming well known! According to various sources, it was on 18th August 2002 that Demo Test 0.04k first became known to more than a few people. It was very exciting to see the reaction to this new racing simulator that seemed to have appeared from nowhere and was quite different to what was available at the time. It had a focus on multiplayer racing. Despite synchronisation issues, it was already a lot of fun to race around Blackwood, with up to 8 drivers, in an XF GTI, XR GT, or XR GT TURBO.
If you'd like to try 0.04k it is available on our downloads archive page. It does not connect to the master server but it is possible to connect to a host directly by IP address.
LFS has come a long way since 2002, with 9 track areas now available and a huge variety of cars created by racers using the new mods system. Of course, there is more to come, notably a new graphics system and new tyre physics. The new graphics system includes dynamic lighting and day to night transitions. The tyre physics update is more realistic and accurate for a better driving experience.
We decided to show a few pictures of some track updates that Eric has been working on and share some technical details about the ongoing development in the physics system.
Please visit the 20th Anniversary page to see the screenshots, read about the development and watch a few short video clips.
Thank you for all the support over the last 20 years!
- LFS Developers
Odd that two people have actually posted here to deliberately antagonise the developers. It's really strange. Some people should re-evaluate their priorities.
All I can say is there is progress, but we aren't in any hurry to report progress. Eric is working on things and I am working on things. These things take time. We are working several hours every day. Don't hold your breath waiting for updates!
Bugging us will certainly not speed anything up. If anything, it can slow us down a little because sometimes you succeed in irritating us. A happy developer is a productive developer. We have never increased our rate of work as a response to people bugging us. That would be quite impossible as we are working as hard as we can sustain already. No-one wants the updates released more than we do.
This is a huge project we are working on. We work very hard on it and there was a massive update last year, probably the biggest update for years. But some people just want the next thing and are losing patience. It's quite pathetic really but all I can suggest is you go and find a hobby, other than LFS. Do something, maybe get a bike and get outside a bit. Because if you are impatient for updates, it's going to get a lot worse for you before something arrives. Make sure you are subscribed to the newsletter and you will get notified when there is something interesting.
Anyway, I got the message from this thread and will now close it. Thanks for the input!
All I can say is there is progress, but we aren't in any hurry to report progress. Eric is working on things and I am working on things. These things take time. We are working several hours every day. Don't hold your breath waiting for updates!
Bugging us will certainly not speed anything up. If anything, it can slow us down a little because sometimes you succeed in irritating us. A happy developer is a productive developer. We have never increased our rate of work as a response to people bugging us. That would be quite impossible as we are working as hard as we can sustain already. No-one wants the updates released more than we do.
This is a huge project we are working on. We work very hard on it and there was a massive update last year, probably the biggest update for years. But some people just want the next thing and are losing patience. It's quite pathetic really but all I can suggest is you go and find a hobby, other than LFS. Do something, maybe get a bike and get outside a bit. Because if you are impatient for updates, it's going to get a lot worse for you before something arrives. Make sure you are subscribed to the newsletter and you will get notified when there is something interesting.
Anyway, I got the message from this thread and will now close it. Thanks for the input!
I've said before but will explain again as I know not everyone would know this:
The old tyre physics does not exist in the new version of LFS. Too many things were changed - the way the tyres work, the specification of the tyre compounds, the AI drivers, various abstract things, etc. At some point I found it impossible to keep two versions of tyre physics in the one program. There is no file "TyrePhysics.cpp" or something like that. Contributions to the tyre physics are spread around many files and systems within the game code.
So it would be a lot of work to bring the old tyre physics back into the new version of LFS. It's a possible option but would be madness to do that right now. Going back into the past, and making myself suffer for no reason at all.
What seems the right thing to do is try to make the new tyre physics work properly, then all's good and we move into a new era graphically and physically. Anyway Eric still has work to do on various tracks so it's not as if we are just waiting for me. Eric is of course free to comment if he wants to but I don't want to bother him for a graphical progress report at this time. My focus is the tyre physics and that's all really.
The old tyre physics does not exist in the new version of LFS. Too many things were changed - the way the tyres work, the specification of the tyre compounds, the AI drivers, various abstract things, etc. At some point I found it impossible to keep two versions of tyre physics in the one program. There is no file "TyrePhysics.cpp" or something like that. Contributions to the tyre physics are spread around many files and systems within the game code.
So it would be a lot of work to bring the old tyre physics back into the new version of LFS. It's a possible option but would be madness to do that right now. Going back into the past, and making myself suffer for no reason at all.
What seems the right thing to do is try to make the new tyre physics work properly, then all's good and we move into a new era graphically and physically. Anyway Eric still has work to do on various tracks so it's not as if we are just waiting for me. Eric is of course free to comment if he wants to but I don't want to bother him for a graphical progress report at this time. My focus is the tyre physics and that's all really.
I just want to be clear that nearly all the car models were made by Eric.
I only made the UF model. I know it needs some updates.
But I was involved with the physics of all the cars.
In case I get to it at some point, how should dead zones work? I'm thinking if someone could show me a graph of controller input -> deadzone modification -> output that actually works well, it could save me a lot of research.
LFS uses all available axes presented to it by DirectInput. On my computer I have a wireless XBox controller and separate axes are reported. There's some kind of bug (or is it deliberate knobbling) by Microsoft that for some reason does not report separate axes for some gamepads, instead requiring developers to rewrite their controller support software to XInput. So to my mind it is a Microsoft bug. I think it would be a long time before I'm motivated to stop all other work simply to support XInput purely because of Microsofts oversight (or deliberate omission).
I'm sure you must know by now that you can set a controller button to perform any text commands or keypresses? You can set a controller to the text commands like:
/press V
/shift V
/view [fol/heli/cam/driver/custom] - set specific view
That is just fixed at the moment. Eric really just reused a display that was designed for use on the big clock displays at tracks. So that texture shows whatever is displayed at the current track, whether it is race time or UTC time. In fact in the development version it would show the time of day at the selected track (possibly with an offset).
LFS uses all available axes presented to it by DirectInput. On my computer I have a wireless XBox controller and separate axes are reported. There's some kind of bug (or is it deliberate knobbling) by Microsoft that for some reason does not report separate axes for some gamepads, instead requiring developers to rewrite their controller support software to XInput. So to my mind it is a Microsoft bug. I think it would be a long time before I'm motivated to stop all other work simply to support XInput purely because of Microsofts oversight (or deliberate omission).
I'm sure you must know by now that you can set a controller button to perform any text commands or keypresses? You can set a controller to the text commands like:
/press V
/shift V
/view [fol/heli/cam/driver/custom] - set specific view
That is just fixed at the moment. Eric really just reused a display that was designed for use on the big clock displays at tracks. So that texture shows whatever is displayed at the current track, whether it is race time or UTC time. In fact in the development version it would show the time of day at the selected track (possibly with an offset).
Eric said I could do an editor export of his RB4 updates from today so you can check if the bugs you reported have been fixed, in advance of another test patch tomorrow. The veh and steering animation are attached.
This is only for use in the EDITOR!
This is only for use in the EDITOR!
I've sent a message to Eric, in case he thinks it would be better mapped asymmetrically.
Skins and mods use the same http download. People in China have experienced the same problem. I am still baffled why LFS would have trouble downloading relatively small files, when your browser surely wouldn't. As I say, LFS isn't really doing much, but calling 'recv' when there's something to receive. I looked in the code recently but I'm pretty much coming up blank at this point. If anyone is a Winsock / http expert, I'd like to know if there are any great socket settings I should (or should not) use, when initialising a socket for http downloads.
One of the first things I'll do is add more cameras so they aren't quite so extremely zoomed in.
It's actually me who works on this map, in fact Eric originally built this some years back with some distant scenery to make it look nicer, but then got diverted onto all the new graphics stuff before it was finished. Recently I resurrected an old LFS track editor so I could actually save a track that can still be loaded in the public version.
I removed distant scenery as there were problems trying to draw things so far away. And I added the long fences.
I wonder if the configuration system could be used for something useful. I'm not sure we need both wide grid and long grid? Or are they of some use? You can place your own start positions anyway.
I'm pretty busy on other things though. I don't know how easy it would be to allow a configuration to perhaps turn one of the 1km squares into a dirt arena. I could try to copy one or two of Eric's methods and textures and see how it looks.
What would be the purpose of being walled in to a 1km square? As Degats suggests, can't you just use an existing square? Or would it be particularly useful to use the walls as a boundary? Other questions, if I could do a dirt square, would only 1km by 1km be enough? I'm trying to limit both the workload and number of configurations, so rather not do more than is needed.
It's actually me who works on this map, in fact Eric originally built this some years back with some distant scenery to make it look nicer, but then got diverted onto all the new graphics stuff before it was finished. Recently I resurrected an old LFS track editor so I could actually save a track that can still be loaded in the public version.
I removed distant scenery as there were problems trying to draw things so far away. And I added the long fences.I wonder if the configuration system could be used for something useful. I'm not sure we need both wide grid and long grid? Or are they of some use? You can place your own start positions anyway.
I'm pretty busy on other things though. I don't know how easy it would be to allow a configuration to perhaps turn one of the 1km squares into a dirt arena. I could try to copy one or two of Eric's methods and textures and see how it looks.
What would be the purpose of being walled in to a 1km square? As Degats suggests, can't you just use an existing square? Or would it be particularly useful to use the walls as a boundary? Other questions, if I could do a dirt square, would only 1km by 1km be enough? I'm trying to limit both the workload and number of configurations, so rather not do more than is needed.
I kind of think that the tyre physics system is the biggest unknown.
Eric has kept on at South City longer than any of us had ever imagined, because he kept getting new ideas and going for it. I'm guessing he could wrap it up fairly quickly if needed. In my mind Kyoto has the most unfinished parts, though I might be wrong about that.
Eric did leave Kyoto in the end stages when he moved on to South City. But the end stages of projects generally take quite a while. I'm speaking generally, not actually speaking for Eric or specifically about South City or Kyoto.
As I say I believe the tyre physics are still the least predictable. I need to get my teeth into that and see where we are at.
I would like to say yes that is certainly my hope. In my mind, after we get through the public testing of the mods system and it becomes the official version, I should be on the completion of the tyre physics system and Eric will finalise some tracks. So then we can put that together and release the new graphics and tyre physics.
That's how I see it. But plans can change and do change all the time. And apparently every task takes longer than I can possibly imagine. An alternative scenario is to re-implement the old tyre physics into the new graphics system (it was deleted years ago from that branch of the source code). But how much time would be wasted doing that? Maybe it's only a week's work, I don't know. The new tyre physics has a better feeling, which is encouraging as it is based on physical principles. So maybe some good solid work on fixing the link between heating, slip, load, pressure and grip could be sorted out and we'd be ready to go. I prefer this option, though the other way might be quicker to get the graphics to you.
I think I thought the mods support could be done in a few weeks and would be a bit of fun while Eric finished some tracks and I did the tyre physics.
But the mods support turned into a much bigger project. It wasn't an easy task to turn development editors into ones that are suitable for public use, and the way that mods work with automatic in-game download and have version support and so on... these are pretty big systems.
Also it was becoming painfully clear that we needed a way to prevent piracy. Seeing your business melt away due to continual theft isn't a great feeling. Just putting out the updated tracks with the old system would have been a nice gift for the pirates and we simply can't do that any more. Our servers are linked with the mods system too.
May Progress Report
Hello Racers,
It has been a while so we'd like to tell you what we've been doing and some plans. You may have been expecting a report about Fern Bay. But Eric had so many ideas for new ways to link up roads at South City he wasn't ready to stop. The result: more roads and variety than any of us had expected! And I've had a lot to do too. Please follow the link for more information, screenshots and a video!
May Progress Report
We hope you like the pictures and video!
- LFS Developers
It has been a while so we'd like to tell you what we've been doing and some plans. You may have been expecting a report about Fern Bay. But Eric had so many ideas for new ways to link up roads at South City he wasn't ready to stop. The result: more roads and variety than any of us had expected! And I've had a lot to do too. Please follow the link for more information, screenshots and a video!
May Progress Report
We hope you like the pictures and video!
- LFS Developers
FGED GREDG RDFGDR GSFDG